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Abstract. We consider the problem of computing Nash Equilibria of action-graph games (AGGs). AGGs, intro- 
duced by Bhat and Leyton-Brown, is a succinct representation of games that encapsulates both 'local'dependencies 
as in graphical games, and partial indifference to other agents' identities as in anonymous games, which occur in 
many natural settings. This is achieved by specifying a graph on the set of actions, so that the payoff of an agent 
for selecting a strategy depends only on the number of agents playing each of the neighboring strategies in the ac- 
tion graph. We present a Polynomial Time Approximation Scheme for computing mixed Nash equilibria of AGGs 
with constant treewidth and a constant number of agent types (and an arbitrary number of strategies), together with 
| hardness results for the cases when either the treewidth or the number of agent types is unconstrained. In particular, 

, we show that even if the action graph is a tree, but the number of agent-types is unconstrained, it is NP-complete to 

_ ' decide the existence of a pure-strategy Nash equilibrium and PPAD-complete to compute a mixed Nash equilibrium 

(even an approximate one); similarly for symmetric AGGs (all agents belong to a single type), if we allow arbitrary 
treewidth. These hardness results suggest that, in some sense, our PTAS is as strong of a positive result as one can 
expect. 

(N ■ 

i | ' 1 Introduction 

O: 

What is the likely behavior of autonomous agents in a variety of competitive environments? This question has 
q | been the motivation for much of economic theory. Partly due to the increasing prevalence of vast online networks over 

'— 1 • which millions of individuals exchange information, goods, and services, and the corresponding increasing importance 

| | of understanding the dynamics of such interactions, the Computer Science community has joined in the effort of 

j> • studying game-theoretic questions. 

■^j- ' Computing equilibria in games and markets has been extensively studied in the Economics and Operations Re- 

search communities since the 1960's, see e.g. [17,21,26,22,24]. Computational tractability has been recently recog- 
nized as an important prerequisite for modeling competitive environments and measuring the plausibility of solution 
concepts in Economics: if finding an equilibrium is computationally intractable, should we believe that it naturally 
arises? And, is it plausible that markets converge to solutions of computationally intractable problems? Probably not 
. — but if so, we should certainly know about it. 

Computing Nash equilibria in games, even in the case of two players, has been recently shown to be an intractable 
• • . problem; in particular, it was shown to be complete for the class of fixed point computation problems termed PPAD [8, 

. ^ ' 3]. This result on the intractability of computing Nash equilibria has sparked considerable effort to find efficient 

. algorithms for approximating such equilibria, and has increased the importance of considering special classes of games 

^ ' for which Nash equilibria might be efficiently computable. 

For two-player games the hardness of computing approximate equilibria persists even if the required approximation 
is inverse polynomial in the number of strategies of the game [4]; similarly, hardness persists in graphical games if 
the required approximation is inverse polynomial in the number of players [8,4]. The same hardness results apply 
to special cases of the problem, e.g. win-lose games, where the payoff values of the game are restricted to {0, 1} 
[6], sparse bimatrix games, where the number of non-zero entries of each row an column of the payoff matrices is 
a constant, and two-player symmetric games [14]. The emerging question of the research in this field is: Is there a 
Polynomial Time Approximation Scheme (PTAS) for Computing Nash Equilibria? And, which special cases of the 
problem are computationally tractable? 

The zero-sum two-player case is well-known to be tractable by reduction to linear programming [25]. Tractability 
persists in the case of low -rank two-player games, in which the sum A + B of the payoff matrices of the players, instead 
of being 0, has fixed rank; in this case, a PTAS exists for finding mixed Nash equilibria [16]. In n-player graphical 
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games, a PTAS has been provided if the tree-width is O(logn) and the maximum degree is bounded [9]; in the case 
of dense graphical games, a quasi-polynomial time approximation scheme exists [11]. 

An important line of research on tractable special cases has explored games with symmetries. Mutli-player sym- 
metric games with about logarithmically few strategies per player can be solved exactly in polynomial time by re- 
duction to the existential theory of reals [20]. For congestion games, a pure Nash equilibrium can be computed in 
polynomial time if the game is a symmetric network congestion game [13], and an approximate pure Nash equilibrium 
if the game is symmetric but not necessarily a network game and the utilities satisfy a "bounded-jump condition" [5]. 
Another important class of games for which computing an approximate equilibrium is tractable is the class of anony- 
mous games, in which each player is different, but does not care about the identities of the other players, as it happens 
for example congestion games, certain auction settings, and social phenomena [2]; a PTAS for anonymous games with 
a fixed number of strategies has been provided in [10, 11]. For a thorough study of the problem of computing pure 
Nash equilibria in symmetric and anonymous games see [12]. 

In this paper, we consider another special class of games, Action Graph Games (AGGs), that were introduced 
by Bhot and Leyton-Brown [1]. AGGs is a fully general game representation that succinctly captures both 'local' 
dependencies as in graphical games, as well as partial indifference to other agents' identities as in anonymous games. 
Strategies are represented as nodes in a graph, called strategy graph, and the utility of an agent for selecting a strategy- 
node depends on the number of other agents selecting each of the neighboring strategies. The only attribute that 
distinguishes players is the set of strategies that each player is allowed to play. In particular, all agents who play a 
given strategy get the same payoff. A variety of natural games can be concisely represented as AGGs, and we refer the 
reader to [1, 15] for further discussion. 

In the remainder of this section, we discuss previous work on AGGs and summarize our results. In the end of the 
section, we provide definitions. 

1.1 Previous Work 

Action graph games were first defined by Bhat and Leyton-Brown [1] who considered the problem of computing 
Nash equilibria of these games. In particular, they analyzed the complexity of computing the Jacobian of the payoff 
function — a computation that is, in practice, the bottleneck of the continuation method of computing a Nash equi- 
librium. They considered this computation for both general AGGs and AGGs with a single player type (symmetric 
AGGs), and found that this computation is efficient in the latter case. For pure Nash equilibria, Jiang and Leyton- 
Brown [15] show that deciding the existence of such equilibria in AGGs is NP-complete, even in the case of a single 
player type and bounded degree. On the other hand, they provide a polynomial time algorithm for finding pure-Nash 
equilibria in AGGs with constant number of player types when the strategy graph has bounded tree-width. 

1.2 Our Results 

We examine, and largely resolve the computational complexity of computing Nash equilibria in action graph games. 
We give a polynomial algorithm for computing an e-Nash equilibrium for AGGs with constant treewidth and degree 
and a constant number of agent types (and arbitrarily many strategies), together with hardness results for the cases 
when either the treewidth or the number of agent types is unconstrained. In particular, we show that even if the 
strategy graph is a tree with bounded degree but the number of agent types is unconstrained, it is NP-complete to 
decide the existence of a pure-strategy Nash equilibrium and PPAD-complete to compute a mixed Nash equilibrium; 
similarly for AGGs in which all agents are a single type if we allow the strategy graph to have arbitrary treewidth. 
These hardness results suggest that, in some sense, our PTAS is as strong of a positive result as one can expect. While 
Bhat and Leyton-Brown studied heuristics for computing mixed Nash equilibria [1], the authors know of no previous 
complexity theoretic results concerning computing mixed Nash equilibria for AGGs apart from the PPAD-hardness 
result that follows from them being a generalization of normal-form games. 

1.3 Definitions 

In this section we give a formal definition of AGGs and introduce the terminology that will be used in the remain- 
der of this paper. We follow the notation and terminology introduced in [15]. 



Definition 1. An action-graph game, A, is a tuple (P, S, G, u) where 

- P := {1, . . . , n} is the set of agents. 

- S := (Si , . • . , S n ), where Si denotes the set of pure strategies that agent i may play. 

- For convenience, let S := {J i Si = {si, . . . , sigi } denote the set of all strategies, and thus each Si C S. Also, we 
write Si — {s^i, Si t 2, ■ ■ ■ , s i,\S\i} Furthermore, we'll let s(i) denote the strategy played by agent i. 

- For any S' C S, let A(S') denote the set of valid configurations of agents to strategies s G S'; we represent a 
configuration D(S') G A(S') as an \S'\-tuple D(S') = {n\, . . . , where rii is the number of agents playing 
the i th strategy of S 1 . 

- G is a directed graph with one node for each action Si. Let v : S — ► 2 s , be the neighbor relation induced by graph 
G, where s' G v(s) if the edge (s', s) G G. Note that self-loops are allowed, and thus it is possible that s G ^(s)- 
We refer to G as the strategy graph of A. 

- The utility function u assigns identical utilities to all agents playing a given strategy s, with the utility depending 
only on the number of agents playing neighboring strategies. Formally, u : A(S) — * R' s ', via maps U\, . . . , u\g\ 
where Ui : A[v(si)] — ► M defines the common utility of all agents playing strategy Sj. 

Note that AGGs are fully expressive because any games can be written as an action graph game in which the 
strategy sets of different players are disjoint, and the strategy graph G is complete. 

We now define a further type of possible symmetry between agents that will be important in our analysis of the 
complexity of computing Nash equilibria. 

Definition 2. We say that an AGG has k player types ;/ there exists a partition of the agents into k sets Pi, ... , P^, 
such that ifp,p' G Pi, then S p — S p >. (The terminology of [15] refers to such games as k-symmetric AGGs.) 

Since agents who play the same strategy receive the same utility, all agents of a given type are identical — for ex- 
ample an AGG with a single player type is a symmetric game. While the number of player types does not significantly 
alter the description size, decreasing the number of player types constrains the space of possible Nash equilibria; this 
is the motivation for considering AGGs with few player types as a possible class of tractable games. 

A strategy profile, M := [mi, . . . , m n ], with rrii = (pi,i, . . . ,Pi,\St\) assigns to each agent a probability distribu- 
tion over the possible strategies that the agent may play, with Pr[s(z) = s^k] = Pi.k where Sk is the k th element of 
Si. Thus a given strategy profile induces an expected utility for each player E[u|M] = X^Dezi u (^) ^M^)) where 
the probability is with respect to the strategy profile M . 

Definition 3. A strategy profile M is a Nash-equilibrium if no player can increase her expected utility by changing her 
strategy rrii given the strategy profiles m-i of the other agents. That is, for all strategy profiles m^. E[u, |m_i , m.;] > 

Definition 4. A strategy m G M is an e-Nash-equilibrium if no player can increase her expected utility by more than 
e by changing her strategy profile. 

Note that there is the slightly stronger definition of an e-Nash equilibrium in which, for all agents i, the expected 
utility of playing every strategy s in the support of rrii is at most e less than the expected utility of playing a different 
s' G Si. We do not stress the distinction, as our PTAS finds such an e-Nash equilibrium, and our hardness results 
apply to the weaker definition given above. 

2 PTAS 

Action graph games have properties of both anonymous games and graphical games. As such, one might expect that 
classes of AGGs that resemble tractable classes of anonymous or graphical games could have efficiently computable 
equilibria. For anonymous games, the symmetry imposed by the limited number of types implies the existence of a 
highly symmetric mixed equilibrium which seems easier to find than asymmetric equilibria. For graphical games with 
small treewidth, the tree structure allows for an efficient message-passing dynamic-programming approach. For AGGs 
with a bounded number of player types and a strategy graph of constant treewidth, we give a PTAS for computing 
e-Nash equilibria that uses both the symmetry implied by bounding the number of player types, and a dynamic pro- 
gramming approach that exploits the tree structure. While these conditions might seem strong, we show in Section 3 
that if either condition is omitted the problem of computing an e-Nash equilibrium is hard. 



Theorem 1. For any fixed constants d, k, and t, an AGG A with k player types and strategy graph Ga with bounded 
degree d and treewidth t, an e-Nash equilibrium can be computed in time polynomial in \A\, 1/e. 

We begin with a fact about games with few player types. 

Fact 2 [18] Any AGG with k player types has a Nash equilibrium where all players of a given type play identical 
mixed strategy profiles. Formally, there is a strategy profile M — [mi, . . . , m n ] such that if Si = Sj, then mi = rrij. 
We refer to such equilibria as type-symmetric equilibria. 

The the high-level outline of the PTAS is as follows: we discretize the space of mixed strategy profiles such that 
each player may play a given strategy with probability N5 for N £ N, and some fixed S > that will depend on 
e and n. We also discretize the space of target expected utilities into the set V = {0, e/2, e, . . . , 1}. Then, for each 
i E {0, . . . , |V|}, starting from the leaves of the strategy-graph tree, we employ dynamic programming to efficiently 
search the discretized strategy space for a type-symmetric e-Nash equilibrium in which each strategy in the support has 
an expected utility close to Wj. To accomplish this we associate to each strategy s; a polynomially sized table expressing 
the set of probabilities with which Sj could be played so that some assignment of probabilities to the strategies below 
Si in the strategy tree could be extended to such an e-Nash equilibrium for the whole game. The following lemma 
guarantees the existence of such a type-symmetric e-Nash equilibrium. 

Lemma 1. Given an n-player AGG A, with 1 player type and strategy graph Ga with maximum degree d, for any 
5 > there is a strategy profile Q = (qi, ■ ■ . , Q\s\) with each qi a multiple of 5 and the property that if all agents play 
profile Q, for any strategy s in the support of Q, W,[u s \Q] > E[u s /|Q] — 25dnfor all s' G S. 

The following standard fact will be necessary in our proof: 

Fact 3 For binomially distributed random variables X = B(n,p), Y = B(n,p + 5) 

max | Pt(X = k) - Pr(Y = k)\ < nS. 

Proof of Lemma 1: From Fact 2, there exists a strategy profile P = (pi, . . . ,p\s\) which is a Nash equilibrium of A. 
Consider a strategy profile Q with the property that qi = if pi =0, and otherwise \qi — Pi\ < S. (Note that such a 
profile clearly exists. )strategy s, with v(si) = d, we now show that 

|E[u a |Q]-E[u 8 |P]| < Sdn, 

from which our lemma follows. 

For a single neighbor Sj of Sj, from Fact 3 \Pr(D(sj) = k\Q) — Pr(D(sj) = k\P)\ < 5n, and thus if we were 
to replace pj by qj in profile P, this change would affect the expectation of playing by at most 5n. Applying this 
reasoning to each of the d neighbors completes our proof. ■ 

We now describe the PTAS; for clarity we describe the algorithm in the case that k = 1, and Ga is a tree with 
maximum degree 3, although it extends easily to a constant number of player types and constant treewidth. The 
following definition simplifies our description of the algorithm. 

Definition 5. We say that some set of strategy profiles is an e-partial equilibrium for a subset S' C S of strategies if 
for all strategies Sj played with nonzero probability, the expected utility of playing Sj is at most e less than the expected 
utility of playing some other s' £ S'. 

Consider a fixed e > 0, and an n-player AGG A = (P, S, G, u) with k types, with strategies S = {si, . . . , sigi} 
and strategy graph Ga that is a tree with maximum degree of 3. Arbitrarily choose some strategy with degree 1 as the 
root of Ga, and without loss of generality denote it by si. Given a strategy s;, let Sru^, s^(j) denote the right and left 
children of Sj in the strategy graph. If s; has only one child, let denote this child and Su^ = null. Fix 5 = 
Set V := {v , . . . , W|v|} where v t := ^. 

Let fi represent a table of size 4k\V\ = poly(^,n, \S\) associated with strategy Si, which can be thought of as a 
function fi M .ljxV^ {0, 1} where Is = {0, 5,26, ... , 1}. The function fi(p,PR,PL, w, Vi) will indicate whether 
there is a type-symmetric | -partial equilibrium in the set of strategies below Si in Ga with expected utility near Vi and 
strategies Sj, Sru\ , sui) played with probabilities p, pr,pl, respectively, where the probability of choosing strategy 



Si, or one that lies below Sj is w. In addition to fi, we also construct another table <?j : if x V — > /| that will facilitate 
the reconstruction of an e-Nash equilibrium after having computed all the tables /j and gi. The will record the total 
weight used in each of the partial solutions which were combined. A given element of gi will never be used if the 
corresponding element of fa is 0, and for simplicity we neglect to define gi for these entries. Starting from the leaves, 
we calculate the tables fi, gi as follows: 

- if Si is a leaf, /j(p, Pr,Pl, w, u<) = 1 iff p = w and p R = p L = 

- if Si ^ Si has one child, fi(p,Pn,PL, w,v) = 1 and gi(p,pR,pL,w, v) = (w — p, 0) iff pl = and there exist 
Qb,,Ql,w' € Is such that the following conditions hold: 

• fR(i)(pR,lR,qL,w',v) = 1 

• if pr > the expected utility of playing sr^ is in (v — §,d + f ) given that Sj, Sr^r^, S£(r(j)) are played 
with respective probabilities p, g^, g^. 

• if pr = the expected utility of playing sr^ is at most v + | given that s,, Sr/ri^, SL(i?(i)) are played with 
respective probabilities p, qR, q^. 

• w = w' + p 

- if Si 7^ ,s nas two children, p£, w,v) = landgi(p,pR,pL,w,v) = (w R , w L ) if there exist q R ,q R ,w R ,q^ 
qL,w L S Is such that the following conditions hold: 

• fR(i)(PR,q R ,qL, wR ' v ) = 1 = fui)(PL,qR,qt,w L ,v) 

• if Pr > the expected utility of playing sr^ is in (v — §, v + §) given that s,, Sr^u^ , suru)) are played 
with respective probabilities p, g_R, g^. Analogously for the utility of playing SL(i) if Pl > 0. 

• if pr = the expected utility of playing Sru-* is at most v + | given that s,-, s_R(_R(j)), s l ^ r ^ are played with 
respective probabilities p, q R ,q R . Analogously for the utility of playing s^u^ if pl = 0. 

• w = w R + w L + p 

Note that there may be multiple choices of q R , q R ,w R , qR,q^, w L that satisfy the above conditions, in which case 
9i(p,Pr,Pl, w, v) = (w R , w L ) can be assigned to an arbitrary choice of such w R , w L . 

- set Ji(p,Pr,Pl, w,v) = 1 iff Pl = 0, w = 1, and there exist g^, g^, w' E Is such that the following conditions 
hold: 

• fR(i)(pR,qR,qL,w',v) = l 

• if pr > the expected utility of playing s fl m is in (u — |, ?; + |) given that si, s,r(,r(i)), sl(.r(i)) are played 
with respective probabilities p,qR,qi. 

• if pr = the expected utility of playing s_r(i) is at most v + | given that si, s_r(_r(x)) , sl(^(i)) are played 
with respective probabilities p, g_R, gL- 

• if si > the expected utility of playing si is in (w — |, v + |) given that Sfl(i) is played with probability p#. 

• if si = the expected utility of playing si is at most v + | given that s^n ) is played with probability pr. 

• w = w' + p 

The following lemma ensures that the tables f behave as hoped, and we find at least one approximate Nash 
equilibria. The proof follows from the definition above and induction on the tree structure. For the sake of brevity we 
omit a formal proof. 

Lemma 2. Given a strategy profile (gi, . . . , g|s|) that is a j-Nash equilibrium of the form guaranteed in Lemma 1, 
then for all i, fi(qt, qR(i), g_L(i), w,v) = 1, where v is chosen to be a multiple ofe/2 and to be within e/2 of the expected 
utility of playing any strategy in the support, and w is the sum of the weights on strategy i and its descendants in the 
tree. 

The following two lemmas demonstrate that the tables fi can be computed efficiently, and that given the tables, an 
e-Nash equilibrium can be efficiently computed. 

Lemma 3. The tables /o, . . . , f\s\ and go, ■ ■ ■ , g\s\ can be computed efficiently. 

Proof. The size of each table is polynomially sized, f and gi can be computed efficiently given the tables Jr^), 
Lemma 4. Given the tables fo, ■ . ■ , f\s\> an d e-Nash equilibrium can be found efficiently. 



Proof. Starting from the root of the tree Ga, we will populate a strategy profile (pi,... ,P\s\) that will t> e a type- 
symmetric e-Nash equilibrium. Lemmas 2 and 1 guarantee that there will be some choice of p,pr <E Is, and v <E V 
such that Ji(p,pr, 0, 1, v) = 1. We set p\ = p, Pr(\) = Pr, and 'pass' w = 1 — pi and v to strategy s R ^y For all 
other strategies Sj, with i > 2 such that pi has already been fixed but Pru),PlU) have not been fixed yet, the parent 
of Si will have passed a pair w, v. In the case that s, has one child, from our construction of it follows that there 
must be a choice of qR £ /a such that /,(p,, <7,r, 0, w — pi, v) = 1, We then set p.R(i)) = and Pz,m = <Zl and pass 
the pair no' = w — pi , v to Sr^ . In the case that Sj has two children, from our construction of /j, it follows that there 
must be a choice of qR, qL € /a such that fi(pi,qR, qL, w, v) = 1, and gt(pi, qR, qL, w, v) = (w R ,w L ). We then set 
PRU)) = iRi PlU)) = Ql, and pass the pairs w R ,v and w L ,v to the right and left children, respectively. 

From our construction, it follows that Y^iPi = 1> an( l that for every strategy in the support, the expected utility of 
playing that strategy is at most e less than the expected utility of playing any other strategy. In particular, (pi, . . . ,P\s\) 
is an e-Nash equilibrium. 

This algorithm and proof easily extends to the case where there are a constant k player types: simply create tables 
and gf' for each type j : 1 < j < fc and strategy « : 1 < J < |5| and proceed analogously but additionally 
require that fl(pi,PR,PL,w^ ,V^) = if i ^ 5j and p» 7^ 0. That is, enforce that each player type only play the 
strategies available. 

Additionally, using a standard technique, the algorithm extends to the case where the tree-width is bounded by 
some constant t. Intuitively, in this case the strategy graph decomposes into a tree over cliques of size t of vertices on 
the graph. All the vertices in each clique are processed simultaneously. Because t is constant, the increase in running 
time is polynomial. 

Finally, one could consider the setting in which there is an unbounded number of player types, but each type 
consists of a connected region of the tree. An analogue of the above algorithm can handle this setting provided not too 
many player types can play any particular strategy. 

Definition 6. Let Si, . . . , Sk be subset of a vertices of a tree T. Define Sf to be smallest connected region ofT such 
that Si C Sf. We define the overlap of Si, . . . , Sk ofT to be maxt£r\i '■ t G Sf\. 

Corollary 1. For any fixed constants c and d, an AGG A with k-player types Si, ... , Sk, and strategy graph Ga 
which is a tree with bounded degree 1, and the overlap of Si, . . . , Sk on Ga is at most c, an e-Nash equilibrium can 
be computed in time polynomial in \A\, 1/e. 

(i) (i) 

Proof. For each strategy i 6 S, define tables fr' and g~ for type j only when i S d| and proceed as in the case of 
k player types. 

3 Hardness Results 



In this section we state and prove our four hardness results. We show that it is (1) NP-complete to decide the 
existence of pure-strategy Nash equilibria, and (2) PPAD complete to approximate general (mixed Nash) equilibria for 
the classes of action graph games that either (a) have action graphs of treewidth 1 or (b) are symmetric (all agents are of 
a single type). Our two hardness results for pure equilibria will come from reductions from the NP-complete problem 
CIRCUITS AT, and follow the approach of [23]. Our hardness results for approximating mixed Nash equilibria are via 
equilibria-preserving gadgets that let us reduce from the PPAD-complete problem of computing equilibria in the class 
of graphical games where the maximum degree is 3 and each player has only two possible strategies. We begin by 
showing that action graph games are in the class PPAD. 

Mapping Action Graph Games to Graphical Games 

We show the following result which reduces the problem of computing a Nash equilibrium of an action graph game to 
the problem of computing a Nash equilibrium of a graphical game. Since the latter is in PPAD [19], it follows that the 
former is in PPAD as well. 



Theorem 4. Any action-graph game A can be mapped in polynomial time to a graphical game Q so that there is a 
polynomial-time computable surjective mapping from the set of Nash equilibria ofQ to the set of Nash equilibria of A. 

Proof. Let us define a bounded division-free straight-line program to be an arithmetic binary circuit with nodes per- 
forming addition, subtraction, or multiplication on their inputs, or evaluating to pre-set constants, with the additional 
constraint that the values of all the nodes remain in [0, 1]. 

We will show that there exists a bounded division-free staight-line program of polynomial size in the description 
of the action graph game which, given a mixed strategy profile M :— {(pi,i, ■ ■ ■ ,Pi,\Si\)}f=i> computes, for every 
agent i, i = 1, . . . ,n, and for every pure strategy Sj, Sj G Si, of that agent, the expected utility that this agent gets for 
playing pure strategy s^. The proof then follows from Theorems 1 and 2 of [7]. 

Without loss of generality, we will show that there exists a straight-line program of polynomial size for computing 
the expected utility of agent 1 for playing pure strategy si i. For this purpose, let A/" := K s m) an d ^ : = ^( !y ( s i,i))- 
Also, for any subset P' C P of the agents, let A pi be the set of valid configurations of the agents of the set P' to the 
strategies in N, represented as |A/]-tuples of numbers; then, for every D 6 Ap>, let Prp> [D] be the probability that 
configuration D arises from the set of agents P', where the measure Prp/ is taken over the mixed strategies of agents 
in P' . Using this notation, the expected payoff of agent 1 for playing sij can be written as follows 

where l Sl 1 is an |A/]-tuple of numbers having a 1 at the coordinate corresponding to strategy si^ and everywhere 
else, and where D + l Sll represents coordinate-wise addition. 

From Equation (1), it follows that, if there is a bounded division-free straight-line program of polynomial size 
which computes the values {Pr p\{i} [D]}d£A p \ {1} , a straight-line program for computing Ui, Sl l can be constructed 
at an additional cost of 0(\Ap\ m |) = 0(\ A\) arithmetic gates. To conclude the proof, we prove the following lemma. 

Lemma 5. Let Pj = {j + 1, . . . , n} C P and suppose that there exists a bounded division-free staight-line program 
computing the values {Prp, [D]}r}^Ap - Also, suppose that the size of this straight-line program is bounded by g. 
Then there exists a straight-line program of size bounded by g + 0(|A| • (\A\ + \S\)) which computes the values 
{Pr PjU{j} [D]} DeApjUU} . 

Proof. Note first that \ Ap jU ^ | = 0(|A|), where \A\ is the description size of the action graph game. Then, for every 
D G Ap. U {fl, it holds that 

Pr p jU {j} [D] = Pr {j} [j plays s j;k ] Pr P . [D] 

+ Yl Pr m [l S3 , fc ]Pr Pj [^'], (2) 

s j)k e Sj nAf, 

D'eA Pj : 
1-,,* +D' = D 

where in the above expression Prp - [D] — 0, if D Ap j . Observe that in (2) the first summation has at most \S\ terms 
and the second at most \A\ terms. Hence, given the values {Prp. [D]}p,£A P . > the above expression can be evaluated 
with 0(| A\ + 15*1) arithmetic operations. The result follows. 

From the lemma above it follows that there exists a straight-line program of size bounded by 0(n-|A|-(|A| + |5|)) = 
0(|A| 3 ) for computing {Pr p\{i}[-D]} De Ap^y This concludes the proof of the theorem, since all the intermediate 
values of the computations described above are in [0, 1]. 

A Copy Gadget 

As a preliminary to the hardness results of the next two subsections, we describe a copy gadget which will prove 
useful in both NP-completeness and PPAD-completeness results. Intuitively, to simulate games G of high treewidth 
by treewidth 1 action graph games H, we create several "copies" of each player, but only one copy of each edge 



relating players, thus ending up with a very "sparse" simulation, whose treewidth we can control. Explicitly, given 
an AGG A, and an agent i whose strategy set consists of the two strategies S 1 , = {fi, U}, our copy gadget will add 
two additional players a, c, of which player c will be the "copy" and player a is an auxiliary player, whose inclusion 
will allow player i's strategies to be disconnected from player c's. We add strategies for a and c that are {f a ,t a } 
and {f c ,t c } respectively, and set the incentives so that in any Nash equilibrium Pr[s(i) = ti] = Pr[s(b) = tf,] (and 
Pr[*(i) = fi] = Pr[s(6) = fi,]). 




Fig. 1. The copy gadget — in any Nash equilibrium Pr[s(&) = /(,] = Pr[s(i) = fi]. 



Definition 7. Given an AGG A = (P, S, G, u), and an agent i with two strategy choices Si = {fi, ti}, we create 
AGG A' = (P', S', G' , u') from A via the addition of a copy gadget on i as follows: 

- P' := PU{a,c}. 

- S' := (Si, . . .,S\ P \,S a , S c ), where S a = {f a ,t a }, and S c = {f c ,t c }, where f a ,t a ,f c ,t c g S. 

- G' consists of the graph G with the additional vertices corresponding to f a ,t a , f c ,t c , and the directed edges 

(fit /a)i Xpai fc): (fci ^a)- 

- v! is identical to u for all strategies in S'\{S a U S c }, and for a configuration D, u'(f a ) = D(fi), u'(t a ) = 
l-D(f c ),andu'(f c ) = l-2D(t a ). 

See Figure 1 for a depiction of the copy gadget. 

Lemma 6. Given an AGG A and an agent i, the addition of a copy gadget on i yields A' that satisfies the following 
properties: 

- The description size of A' is at most a constant larger than A. 

- In the strategy graph Ga>, fc and t c are not path connected to either fi or ti. 

- In every e 2 -Nash equilibrium with agent i's profile (pij, 1 — Pi.f), agent c's profile will have \p c j — Pi,f\ < e 
(and \pb tt - pi.t | < e). 

Proof. The first two properties follow directly from Definition 7. For the third property, assume otherwise and consider 
the case where p c j > e + pij. Agent a's expected utility for playing f a is Pij, and is p c j for playing t a , thus our 
assumption that p c j > e+Pij implies that agent a must be playing t a with probability at least 1 — e since the game is 
at e 2 -equilibrium. Given that a plays t a with probability at least 1 — e, agent b maximizes her utility by playing t c , and 
thus p c j < e, which contradicts our assumption that p c j is the larger of p c ,f,Pi,f, namely at least |. An analogous 
argument applies to rule out the case p c j < pij — e. 

3.1 PPAD-Completeness 



Our PPAD-Completeness results are reductions from the problem of computing equilibria in graphical games, and rely 
on the following fact due to [4] . 



Fact 5 For the class of graphical games with n players, maximum degree 3 and payoffs in {0, 1, 2}, it is PPAD- 
complete to compute e-Nash equilibria where e oc \/poly{n). 

Theorem 6. Computing a Nash equilibrium for AGGs with strategy graph G a is PPAD-complete even if 
treewidth(G a) = L andGA has constant degree. 

Proof. From Theorem 4 this problem is in PPAD. 

To show PPAD-hardness, we reduce from the known PPAD-hard problem of Fact 5. Given an instance of such a 
graphical game H, we construct an AGG A' H with treewidth 1 and maximum degree 4 with similar description size 
to H such that there a polynomial time mapping from e-Nash equilibria of A' H to the e-Nash equilibria of H. We 
construct A' H via the intermediate step of constructing an AGG Ah which will be equivalent to H and might have 
large treewidth. From Ah, we construct A' H using our copy gadget to reduce the treewidth of the associated strategy 
graph. See Figure 2 for a depiction of the reduction. 

The construction of Ah is straightforward: for each player in in the graphical game, we have a corresponding 
player %a in the AGG with strategy set Si A — {/,, U}, corresponding to the two strategies that in may play in H. 
For each undirected edge between players (i, j) S H, we add directed edges between the t nodes (tj, ti), (U, tj), and 
edges between the / nodes (fj, fi), (fi, fj) to the strategy graph Ga h °f 4zj- We define utilities u by simulating the 
utility functions from the original game H: from each / strategy connected to %a in the AGG we know that if it is 
played then the corresponding t strategy is not played and vice versa; thus we have recovered the strategy choice of 
each neighbor of Ih in original graphical game; we then apply the utility function of the graphical game to compute 
the utility in the AGG. We do the symmetric procedure for the t nodes of the AGG. From the construction, it is clear 
that H and Ah represent the same game via the correspondence in — * ia> an d m particular an e-Nash equilibrium of 
one game will correspond to an e-Nash equilibrium of the other game via the natural mapping. 

We obtain A' H from Ah by making three copies of each %a via the copy gadget. Thus for each i there are agents 
iA,iA,i\, i\ w i m Sjk = {fi A >ti A }- Finally, for each of the (at most three) outgoing edges of fo A that are not part of 
copy gadgets, i.e the edges of the form (fi A , fj A ), we replace the edge by (ff A , fj A ), with each fj? having at most one 
outgoing edge, and modify the utility function u analogously so as to have the utility of strategy fj A depend on 
instead of f % A . Analogous replacements are made for the outgoing edges of ti A . Since the copied strategies ff , 
are disconnected from the original strategies f % A , U A the longest path in the strategy graph G a' h associated with A' H 
has length at most 4, with maximum degree 6, and treewidth(G a h ) — L (See Figure 2.) Lemma 6 guarantees that 
the transformation from Ah to A' H increases the representation size by at most a constant factor. Further, from an 
Tjije 2 -Nash equilibrium of A' H we can extract an e-Nash equilibrium of Ah by simply ignoring the new players: 
all of the copies i k A of a player %a will play strategies with probabilities within j^e of the probabilities of playing the 
original by Lemma 6; thus the joint distribution of any triple of these will have joint distribution within ^e of the "true" 
joint distribution; since each utility has magnitude at most 2 the computed utilities will be within |e of the utilities 
computed in Ah', thus each of the mixed strategies of a player %a in A' H , interpreted as a strategy in Ah will yield 
utility within e of optimal. From Fact 5 we conclude that finding an ^e 2 -Nash equilibrium of A' H is PPAD complete 
for any polynomial e, yielding the desired result. 

We now turn our attention to AGGs that have a constrained number of player types. 

Theorem 7. Computing a Nash equilibrium for symmetric AGGs (1 player type) is PPAD-complete even if the strat- 
egy graph G a has bounded degree. 

To show PPAD-hardness, as above we reduce from the known PPAD-hard problem of computing Nash equilibria 
in graphical games of degree at most 3 where each player chooses between 2 strategies /, t and has utility 0, 1, or 2. 
Given such a graphical game H, we will reduce it to an AGG Ah that has strategies f i: i, corresponding to the two 
strategies that agent i may choose in H. Intuitively, if our reduction is to be successful there are several properties of 
Gh that seem necessary. First, in every Nash equilibrium of Gh, there must be at least one agent playing either /j or 
ti for every i. This is accomplished by giving agents a bonus payment if they choose either of the two strategies of a 
sparsely-played fc, U pair. Second, there must be some unambiguous mapping between the number of agents playing 
fi and U in Ah to a choice of actions of agent i in H. This is accomplished via the MAJORITY function: if more 
agents play fi than ti in Ah, we say that i plays /. This motivating intuition is formalized in the proof below. 
Proof of Theorem 7: From Theorem 4 this problem is in PPAD 

To show PPAD-hardness, we reduce from the known PPAD-hard problem of computing Nash equilibria in graph- 
ical games of degree at most 3 where each player chooses between 2 strategies and has utility 0, 1, or 2. Given an 



H 



Fig. 2. The transformation from the graphical game H to the AGG A' H . For simplicity, the internal strategies and edges associated 
with the copy gadgets are omitted. 



instance of such a graphical game H, with n agents, and some e > we construct the symmetric AGG Ah so that an e- 
Nash equilibrium of Ah can be efficiently mapped to a 2e-Nash equilibrium of H. We construct Ah = (P, S, Ga 7 u) 
as follows: 
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- P := {1, . . . ,3cn} with c > 

- S := (S, . . . , S), that is, each player has identical (symmetric) strategy set S := ti, ■ ■ ■ , /«, in} where 
strategies /j and tj correspond to the two strategies of the i th agent of H. 

- For every undirected edge in the graph of H, the strategy graph Ga has the eight directed edges (/,, /j ), 
{fj,fi)> (fi,tj), (tj,fi), (U, fj), (fj,U), (ti,tj), (tj,U). Furthermore, for all i £ {1, . . . , n}, G A contains the 
edges (fi,U), (U, fi) and the self loops (f t , fi) and (t is ti). 

- To simplify the description of the utility function u, it will be useful to define the indicator functions 
h [D{fiM)l • • ■ , In[D(f n , t»)] where 



Ii[D(fi,ti)] := 



f if D{f t ) > D(ti) 
t if D(fi) <D(U) 



Let m assign utility to /, as a function of D(v(i)), where v(i) denotes i's neighbors, by applying the utility function 
for agent i from H on the simulated actions of her neighbors ji,j2, J3 evaluated as Ij 1 [D(fj 1 , tj 1 )], Ij 2 [D(fj 2 , tj 2 )], 
and Ij 3 [D(fj 3 , tj 3 )], respectively. Finally, if D(fA + D(U) < c, u assigns an extra 100 utility to strategies /j and 
U. 

Observe that the description size of Ah is polynomial in an, and thus is polynomial in the description size of H. From 
Fact 5 our theorem will follow if we show that any e 2 -Nash equilibrium of Ah can be efficiently mapped to an e-Nash 
equilibrium of H. 

Consider the map from mixed strategy profiles of Ah to mixed strategy profiles of H given by : Ma — > Mh 
that assigns M H = [(Pi,f, 1 - Pi,/), ■ ■ ■ , (p«,/, 1 - Pn,/)] by setting p hf := Pr MA {Ii = f) where the probability is 
taken over the distribution over A defined by Ma - It is clear that the map <fi can be computed efficiently, as it essentially 
involves simply evaluating multinomial distributions on 4 outcomes. 

Before showing that <f> maps e-equilibria to 2e-equilibria we first show that the "extra utility" of 100 correctly 
incentivizes a large number of players to play on each strategy pair. We observe that in any mixed strategy profile there 
will be at least one agent, j, who has probability at most 1/3 of receiving a payoff of at least 100. Since his payoff 
from the simulation of H is at most 2, such an agent's expected utility is at most 35 + 1, and thus any Nash equilibrium 
mixed strategy profile must satisfy 100 Pr(£J(/j) + D{tA < c) < 36, Vi £ {1, . . . , n}. If this were not the case, then 
agent j could improve her expected utility to at least 36 by always choosing strategy fi, a contradiction. The above 
inequality implies that 

E[max(D(fi),D(ti))} > °-. 



We now proceed with the proof of correctness of the map <j>. Let Ma be an e-Nash equilibrium of Ah, and 
Mh = 4>(Ma ) ■ Consider a player i in the graphical game, and a strategy of his that he plays with probability at least e. 
Without loss of generality let this strategy be /,-. We show that his utility for playing is at least his utility for playing 
his other choice, ti, minus e; taken together, these statements imply that Mh is an e-Nash equilibrium of the graphical 
game, as desired. 

Since E [max (D(fi), D(ti))] > f, we have that if /, is played with probability at least e, namely if Pr(D(f i ) > 
D(U)) > e then (by Chernoff bounds) we must have K[D(fi)] > |. This implies that for at least one of the 3cn 
players j in Ah, his probability of playing f, is at least y|^-, which is at least 2e. Thus, since Ma is, by assumption, an 
e 2 -Nash equilibrium, we have that player j's utility for playing fa is at most |e below his utility for playing ti, when 
the other players play from Ma- Further, by construction, each of these two utilities are within |e of the utilities for 
player i in the graphical game to play fa, ti respectively, when the other players play from Mh- this is because fixing 
player j's move from Maj to one of /,-, tj changes each Ik by at most the probability of j playing one of (fk,tk) 
divided by the standard deviation of max (D(fi), D(ti)), which is at least the square root of its expectation, = |. 
Thus by the triangle inequality we have that player i's utility for playing fa is no more than e worse than playing tj. 
Since our choice of i was arbitrary and a corresponding argument applies to the t strategies, we conclude that Mh is 
an e-Nash equilibrium of H, as desired. ■ 

3.2 NP-Completeness 

Both of our NP-completeness results are reductions from the NP-Complete problem CIRCUITSAT and follow an 
approach employed in [23]. 

Fact 8 It is NP-complete to decide satisfiability for the class of circuits consisting of AND, OR, and NOT gates, with 
maximum degree 3 (in-degree plus out-degree). 

In our reductions from CIRCUITSAT, given a circuit C, we construct an AGG Ac that computes C in the sense 
that pure strategy Nash equilibria of Ac map to valid circuit evaluations. To this game we add two agents that have a 
simple pure-strategy equilibrium if C evaluates to true, but when C evaluates to false play pennies — a simple game 
that has no pure strategy Nash equilibria. Thus the existence of a pure strategy Nash equilibrium is equivalent to the 
satisfiability of C. 

Theorem 9. Deciding the existence of a pure strategy Nash equilibrium for AGGs with strategy graph G a is NP- 
complete even if treewidth(G a) = h and G a has constant degree. 

Proof. Membership in NP is clear. To show hardness, given a circuit C, we construct the associated AGG Ac ■= 
(P, S, Ga, u) as follows: 

- P := {1, . . . , n,pi,p2}, where n is the number of gates in C, and the gate corresponding to player n is the output 
gate. 

- S . ((/l i t\ ),...,(/„, in), { fpi 1 tpi ) J (fp2 i tp2 ))■ 

- For every pair of gates i, j for which the output of gate i is an input to gate j, Ga has the edges (f t , /_,), and 
(fi,tj). Furthermore, we add edges (f n J Pl ), (fn,t Pl ), (f n , f P2 ), (f n , t P2 ), and the edges (f Pl ,f P2 ), (f Pl ,t P2 ), 

(Jp2 1 I Pi )' (fp2 >tpi)' 

- The utility function u is defined as follows: if agent i corresponds to an input gate, than strategies U both have 
utility 0. For any other agent i corresponding to a gate of C, the payoff of strategy f i is 1 or according to whether 
fi is the correct output value of gate i given the values corresponding to the strategies played by neighboring 
agents/strategies. Similarly for the payoff for strategy ti. If D(f n ) = 0, then f Pl and t pi have utility 0, otherwise 
the utility of pi is 1 if D(f Pl ) = D(f P2 ), and is otherwise. The utility of p2 is 1 if D(f Pl ) ^ D(f P2 ), and is 
otherwise. 

From the construction it is clear that if C is satisfiable, there is a pure strategy profile for agents 1, . . . , n with agent n 
playing t n , such that agents 1, . . . ,n can not improve their utility by deviating from their strategies. Furthermore, p\ 
will be indifferent between her strategies, and pi will play the opposite of p\, in particular, there will be a pure strategy 
Nash equilibrium. If C is not satisfiable, then any pure strategy profile that is an equilibrium for agents 1, . . . , n will 
have D(f„) = 1, and thus p\ will be incentivized to agree with p2, and p2 will be incentivized to disagree, and thus 
Ac will admit no pure strategy Nash equilibrium. 



To complete the proof, note that we can apply the copy gadget to each agent of Ac, as was done in the proof 
of Theorem 6 to yield the game A' c that has strategy graph of treewidth 1, and a mapping from equilibria of A' c to 
equilibria of Ac- 

Theorem 10. Deciding the existence of a pure strategy Nash equilibrium for symmetric AGGs (1 player type) is 
NP -complete even if the strategy graph G a has bounded degree. 

Proof. Membership in NP is clear; to show hardness we proceed as was done in the proof of Theorem 9, and obtain 
AGG Ac from circuit C. Now, we make Ac symmetric by retaining the same number of agents, but allowing each of 
them to pick any of the strategies. We modify the strategy graph G by adding edges (f x ,t x ), (t x , f x ), (f x , f x ), (t x ,t x ) 
for each player x from Ac, and extend the utility function u so that if D(f x ) + D(t x ) > 1 then strategies f x and t x 
have utility — 1. Thus in any pure strategy Nash equilibrium D(f x ) + D(t x ) = 1, and the reasoning in the proof of 
Theorem 9 applies to complete our reduction. 

4 Conclusions and Open Problems 

We have presented results that further the understanding of the computational complexity of computing approximate 
Nash equilibria in action graph games. We provided a polynomial-time algorithm for finding approximate Nash equi- 
librium in action graph games under the restrictions that 1) the strategy graph of the game has constant degree and 
constant treewidth and 2) the players can by classified by a constant number of types. We showed that restricting the 
tree-width and the number of types of players is necessary to avoid the problem becoming PPAD-hard. Whether or 
not the restriction on the degree of the graph is necessary remains an open problem for future study. 

We further showed the our algorithm extends to the case where the underlying graph is a bounded-degree tree 
and there are an arbitrary number of player types, but 1) each player type is a connected region of the graph and 2) 
each strategy is available to only a constant number of types. It remains an open problem if the second restriction is 
required. Furthermore, perhaps there are other restricted classes of games that circumvent the hardness result while 
retaining some of the motivating features of general action graph games. 

We also study the complexity of computing pure Nash equilibria in action graph games with fc-types. While Jiang 
and Brown in [15] show the problem is tractable provided 1) k is constant and 2) the graph has bounded tree width, 
we show that both of these restrictions are necessary. Without either of them, finding pure Nash equilibrium in action 
graph games becomes iVP-complete. 
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